Remove unneeded exit conditions for parse_stream#3916
Merged
Conversation
As I understand it, the code operates line-by-line. When finding `__END__`, it has to decide if it is the end marker or just appears as part of of a multi-line literal (see tests). For a future PR I plan to make, `lex_modes` may be empty after parsing finished, even if it was syntax invalid. Currently there will _always_ be errors when lex modes are not popped. So these extra checks are redundant. Basically, if we found `__END__` on a line by itself and there are syntax errors, it's impossible to be the end marker.
Earlopain
force-pushed
the
parse-stream-end-condition
branch
from
be3da9d to
8b63780
Compare
Collaborator
Author
|
I tested with the examples in #1247 and it seems ok. There is a difference for the following code: 1 +
__END__
foo# Before (source stopped directly after __END__)
$ cat ../test.rb | ruby
-: -:3: syntax errors found (SyntaxError)
1 | 1 +
2 |
> 3 | __END__
| ^~~~~~~ unexpected end-of-input, assuming it is closing the parent top level context
| ^~~~~~~ unexpected end-of-input; expected an expression after the operator
# After (source is now the entire code)
$ cat ../test.rb | ./miniruby
-: -:3: syntax errors found (SyntaxError)
1 | 1 +
2 |
> 3 | __END__
| ^~~~~~~ unexpected end-of-input, assuming it is closing the parent top level context
| ^~~~~~~ unexpected end-of-input; expected an expression after the operator
4 |
5 | foo
This however now matches the output when reading the code from file. So it seems fine. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As I understand it, the code operates line-by-line. When finding
__END__, it has to decide if it is the end marker or just appears as part of of a multi-line literal (see tests).Basically, if we found
__END__on a line by itself and there are syntax errors, it's impossible to be the end marker.This is in preparation for #3911. That issue is caused by
lex_modesnot being properly reset after reaching EOF during heredoc body parsing. It is still inPM_LEX_EMBEXPRmode, which causes it to not reset the parser to the heredoc identifier end.Tests are just formatted so it's better to see what they are doing.